Object Recognition Research Articles

The effect of magnesium sulfate on memory and anxiety-like behavior in a rat model: an investigation of its neuronal molecular mechanisms

ABSTRACT Background Anxiety is an adaptive response to potentially threatening conditions. Excessive and uncontrolled anxiety responses become nonadaptive and cause anxiety disorders. To better understand the anxiety-modulating effects of Mg sulfate, behavioral test batteries in the assessment of anxiety and learning and memory functions were performed simultaneously over a time period. This study also examines the effects of Mg sulfate compared to diazepam, an anxiolytic drug with amnestic effects on anxiety-like behavior, as well as possible oxidative-nitrosative stress and hippocampal changes in male rats exposed to predator odor. Methods Young adult Sprague-Dawley male rats were used. The rats were assessed using a comprehensive neurobehavioral test battery consisting of novel object recognition, open field, and successive alleys tasks. Anxiety was induced by cat odor, and diazepam and Mg were used as study drugs. Of the frontal cortex and hippocampus, the state of total oxidant and antioxidant and NO levels and histological examination of hippocampal CA1, CA2, CA3, and DG regions were performed. Results Diazepam- and Mg-treated rats showed an improvement in anxiety-related behavior to predator odors. Furthermore, Mg treatment alleviated some of the increasing oxidative stress in the frontal cortex and hippocampus of rats, while diazepam treatment in particular enhanced hippocampal oxidant and antioxidant activity. In addition, brain NO increase induced by animal odor exposure or diazepam treatment was ameliorated by Mg administration. Conclusions Overall, our work suggests that Mg had a partial anxiolytic effect on anxiety-like behaviors, although not as much as diazepam, and this effect varied depending on the dose. Mg treatment might counteract increased oxidative stress and elevated NO levels in the brain.

A texture feature extraction method considering spatial continuity and gray diversity

Texture features play an important role in the field of remote sensing classification. However, most existing methods lack a comprehensive consideration of spatial continuity, which makes them either destroy the spatial integrity of regular ground objects or fail to quantify the fragmentation degrees of irregular ground objects. These problems weak the ability of existing methods to distinguish ground objects with different fragmentation degrees. Therefore, this study proposed a new texture feature extraction method considering spatial continuity and gray diversity (SCGD). SCGD first connected all pixels in a neighborhood in series from end to end according to the row and column directions, and the diversities of the spatial continuity encoding in different directions were calculated by the Shannon index. Then, the Shannon index was used to calculate the gray diversity. Finally, SCGD calculated the weighted average of spatial continuity diversity and gray diversity to obtain the final texture feature values. Validation results indicated that SCGD can effectively distinguish ground objects with different fragmentation degrees, and its performance is better than that of traditional methods. As the spatial resolution decreases, its performance advantage becomes more obvious. Moreover, SCGD has great application potential in the field of ground object classification, and combining it with deep learning models will contribute to achieving the fine recognition of ground objects.

Monitoring through AI Based Remote Access Vehicle in Hydro Power Plant

The development of hydroelectric power is essential to the production of sustainable energy. Continuous monitoring and maintenance are necessary to guarantee the safe and effective functioning of hydroelectric plants. A revolutionary method for improving the security, effectiveness, and monitoring capacities of hydroelectric power plants is presented in the project "Monitoring through AI-Based Remote Access Vehicle in Hydro Power Plant."This creative project uses cutting-edge technologies, such as Bluetooth connectivity, artificial intelligence (AI), Raspberry Pi, and sensor systems, to develop a remotely controlled car that is specifically designed to meet the demands of hydroelectric power plant conditions. Real-time object detection and obstacle avoidance are made possible by this configuration, guaranteeing safe passage across the complex industrial infrastructure.Incorporating a temperature sensor also makes it easier to identify fire situations early on, while a water-detecting sensor protects against water leaks. By utilizing AI algorithms for object recognition, vehicles can detect and react to impediments in a proactive manner, hence decreasing the likelihood of accidents. As sentinels, the temperature and water leakage sensors keep an eye on things constantly and send out alarms right once if anything seems out of the ordinary. Reduced reliance on in-person inspections due to remote monitoring capabilities results in lower costs and more dependability. To sum up, this initiative tackles important issues pertaining to hydro power plant maintenance, efficiency, and safety

Effects of Intravenously Administered Plasma from Exercise-Trained Donors on Microglia and Cytokines in a Transgenic Rat Model of Alzheimer’s Disease

Background: Microglia and inflammation play a significant role in Alzheimer’s disease (AD). Physical exercise and peripheral signals can influence microglial activity in the brain. Modulating the inflammatory response in the brain may provide therapeutic approaches for AD. Objective: To assess the effects of intravenously administered blood plasma from exercise-trained donor rats on cognitive function, microglia, and cytokine levels in an AD rat model at two different pathological stages; an early pre-plaque stage and a later stage closer to the emergence of extracellular plaques. Methods: Male transgenic McGill-R-Thy1-APP rats aged 2 and 5 months received 14 injections over 6 weeks: 1) plasma from exercise-trained rats (ExPlas), 2) plasma from sedentary rats (SedPlas), or 3) saline. Cognitive function was evaluated in a novel object recognition task. Microglia count and morphology were analyzed in cornu ammonis, dentate gyrus, entorhinal cortex, and subiculum. Amyloid plaque number and size were assessed in the rats with the later treatment start. A multiplex assay was used to measure 23 cytokines in cornu ammonis. Results: In rats treated from 2 months of age, ExPlas and SedPlas increased number and length of microglial branches in cornu ammonis and dentate gyrus compared to saline. Only ExPlas-treated rats exhibited similar changes in subiculum, while entorhinal cortex showed no differences across treatments. Microglia count remained unaffected. In rats treated from 5 months of age, there were no significant differences in microglia count or morphology or the number or size of amyloid plaques in any brain region. Compared to both other treatments in early pre-plaque stage rats, SedPlas increased TNF-α levels. ExPlas upregulated GM-CSF, IL-18, and VEGF, while SedPlas increased IL-10 compared to saline. In later-stage rats, ExPlas upregulated IL-17, and SedPlas upregulated TNF-α compared to saline. There were no effects of treatments on recognition memory. Conclusions: Intravenous injections of blood plasma from exercise-trained and sedentary donors differentially modulated microglial morphology and cytokine levels in the AD rat model at an early pre-plaque stage of pathology. Exercised plasma may reduce proinflammatory TNF-α signaling and promote microglial responses to early Aβ accumulation but the lack of treatment effects in the later-stage rats emphasizes the potential importance of treatment timing.

Energy Efficiency Optimisation of Joint Computational Task Offloading and Resource Allocation Using Particle Swarm Optimisation Approach in Vehicular Edge Networks

With the progression of smart vehicles, i.e., connected autonomous vehicles (CAVs), and wireless technologies, there has been an increased need for substantial computational operations for tasks such as path planning, scene recognition, and vision-based object detection. Managing these intensive computational applications is concerned with significant energy consumption. Hence, for this article, a low-cost and sustainable solution using computational offloading and efficient resource allocation at edge devices within the Internet of Vehicles (IoV) framework has been utilised. To address the quality of service (QoS) among vehicles, a trade-off between energy consumption and computational time has been taken into consideration while deciding on the offloading process and resource allocation. The offloading process has been assigned at a minimum wireless resource block level to adapt to the beyond 5G (B5G) network. The novel approach of joint optimisation of computational resources and task offloading decisions uses the meta-heuristic particle swarm optimisation (PSO) algorithm and decision analysis (DA) to find the near-optimal solution. Subsequently, a comparison is made with other proposed algorithms, namely CTORA, CODO, and Heuristics, in terms of computational efficiency and latency. The performance analysis reveals that the numerical results outperform existing algorithms, demonstrating an 8% and a 5% increase in energy efficiency.

MDPV (3,4-methylenedioxypyrovalerone) administered to mice during development of the central nervous system produces persistent learning and memory impairments.

Synthetic cathinones (SC) constitute the second most frequently abused class of new psychoactive substances. They serve as an alternative to classic psychostimulatory drugs of abuse, such as methamphetamine, cocaine, or 3,4-methylenedioxymethamphetamine(MDMA). Despite the worldwide prevalence of SC, little is known about their long-term impact on the central nervous system. Here, we examined the effects of repeated exposure of mice during infancy, to 3,4-methylenedioxypyrovalerone(MDPV), a SC potently enhancing dopaminergic neurotransmission, on learning and memory in young adult mice. All experiments were performed on C57BL/6J male and female mice. Animals were injected with MDPV (10 or 20mg/kg) and BrdU (bromodeoxyuridine, 25mg/kg) during postnatal days 11-20, which is a crucial period for the development of their hippocampus. At the age of 12weeks, mice underwent an assessment of various types of memory using a battery of behavioral tests. Afterward, their brains were removed for detection of BrdU-positive cells in the dentate gyrus of the hippocampal formation with immunohistochemistry, and for measurement of the expression of synaptic proteins, such as synaptophysin and PSD95, in the hippocampus using Western blot. Exposure to MDPV resulted in impairment of spatial working memory assessed with Y-maze spontaneous alternation test, and of object recognition memory. However, no deficits in hippocampus-dependent spatial learning and memory were found using the Morris water maze paradigm. Consistently, hippocampal neurogenesis and synaptogenesis were not interrupted. All observed MDPV effects were sex-independent. MDPV administered repeatedly to mice during infancy causes learning and memory deficits that persist into adulthood but are not related to aberrant hippocampal development.

Procedure for describing traffic situation scene development

PurposeThis study aims to comprehensively address the challenge of delineating traffic scenarios in video footage captured by an embedded camera within an autonomous vehicle.Design/methodology/approachThis methodology involves systematically elucidating the traffic context by leveraging data from the object recognition subsystem embedded in vehicular road infrastructure. A knowledge base containing production rules and logical inference mechanism was developed. These components enable real-time procedures for describing traffic situations.FindingsThe production rule system focuses on semantically modeling entities that are categorized as traffic lights and road signs. The effectiveness of the methodology was tested experimentally using diverse image datasets representing various meteorological conditions. A thorough analysis of the results was conducted, which opens avenues for future research.Originality/valueOriginality lies in the potential integration of the developed methodology into an autonomous vehicle’s control system, working alongside other procedures that analyze the current situation. These applications extend to driver assistance systems, harmonized with augmented reality technology, and enhance human decision-making processes.

A Novel Approach to Optimizing Convolutional Neural Networks for Improved Digital Image Segmentation

To divide a digital image into individual parts that share similar characteristics is known as digital image segmentation, and it is a vital research subject in the field of computer vision. Object recognition, medical imaging, surveillance, and video processing are just a few of the many real-world contexts where this study could prove useful. While digital image segmentation research has come a long way, there are still certain obstacles to overcome. Segmentation algorithms frequently encounter challenges in achieving both accuracy and efficiency when confronted with intricate settings, noisy pictures, or fluctuating lighting conditions. The absence of established evaluation standards adds complexity to the process of performing equitable comparisons among different segmentation methodologies. Due to the subjective nature of photo segmentation, attaining consistent results among specialists can be challenging. The integration of machine learning and deep neural networks into segmentation algorithms has introduced new challenges, including the need for large amounts of annotated data and the interpretability of the outcomes. Given these challenges, the objective of this study is to enhance the segmentation model. To this end, this research suggests a model of convolutional neural networks that is optimal for digital picture segmentation. The model is based on a dense convolution neural network, and it incorporates a transfer learning technique to significantly boost the model’s robustness and the quality of picture segmentation. The model’s adaptability to new datasets is improved by the incorporation of a transfer learning method. As demonstrated by experimental results on two publicly available datasets, the suggested methodology considerably enhances the resilience of digital picture segmentation.

Online elasticity estimation and material sorting using standard robot grippers

AbstractStiffness or elasticity estimation of everyday objects using robot grippers is highly desired for object recognition or classification in application areas like food handling and single-stream object sorting. However, standard robot grippers are not designed for material recognition. We experimentally evaluated the accuracy with which material properties can be estimated through object compression by two standard parallel jaw grippers and a force/torque sensor mounted at the robot wrist, with a professional biaxial compression device used as reference. Gripper effort versus position curves were obtained and transformed into stress/strain curves. The modulus of elasticity was estimated at different strain points and the effect of multiple compression cycles (precycling), compression speed, and the gripper surface area on estimation was studied. Viscoelasticity was estimated using the energy absorbed in a compression/decompression cycle, the Kelvin-Voigt, and Hunt-Crossley models. We found that (1) slower compression speeds improved elasticity estimation, while precycling or surface area did not; (2) the robot grippers, even after calibration, were found to have a limited capability of delivering accurate estimates of absolute values of Young’s modulus and viscoelasticity; (3) relative ordering of material characteristics was largely consistent across different grippers; (4) despite the nonlinear characteristics of deformable objects, fitting linear stress/strain approximations led to more stable results than local estimates of Young’s modulus; and (5) the Hunt-Crossley model worked best to estimate viscoelasticity, from a single object compression. A two-dimensional space formed by elasticity and viscoelasticity estimates obtained from a single grasp is advantageous for the discrimination of the object material properties. We demonstrated the applicability of our findings in a mock single-stream recycling scenario, where plastic, paper, and metal objects were correctly separated from a single grasp, even when compressed at different locations on the object. The data and code are publicly available.

AI-Driven Sensing Technology: Review

Machine learning and deep learning technologies are rapidly advancing the capabilities of sensing technologies, bringing about significant improvements in accuracy, sensitivity, and adaptability. These advancements are making a notable impact across a broad spectrum of fields, including industrial automation, robotics, biomedical engineering, and civil infrastructure monitoring. The core of this transformative shift lies in the integration of artificial intelligence (AI) with sensor technology, focusing on the development of efficient algorithms that drive both device performance enhancements and novel applications in various biomedical and engineering fields. This review delves into the fusion of ML/DL algorithms with sensor technologies, shedding light on their profound impact on sensor design, calibration and compensation, object recognition, and behavior prediction. Through a series of exemplary applications, the review showcases the potential of AI algorithms to significantly upgrade sensor functionalities and widen their application range. Moreover, it addresses the challenges encountered in exploiting these technologies for sensing applications and offers insights into future trends and potential advancements.

All‐Optically Triggered In‐Sensor Collision Detection and Warning Based on 2D Complementary Material Devices

AbstractPrecise and timely collision detection and warning are essential to ensure the safety of autonomous driving. However, existing collision detection systems based on image sensors and radars are prone to misjudgment in adverse environments such as darkness or rain. The lobula giant movement detector (LGMD) neuron found in locusts achieves potential collision detection in unpredictable environments without the need for object recognition algorithms. Existing artificial collision detectors inspired by LGMD suffer from complex device structures and sophisticated operating modes. Here, an LGMD‐inspired all‐optically triggered in‐sensor collision detector is presented by 2D complementary material devices (2D‐CMDs) composed of n‐type molybdenum disulfide (MoS2) and p‐type platinum diselenide (PtSe2) connected in series. The proposed 2D‐CMDs couple the positive photoconductivity of MoS2 and negative photoconductivity of PtSe2 in response to looming light, successfully mimicking the antagonism of excitatory and inhibitory responses in LGMD neurons to generate a nonmonotonic escape response. The 2D‐CMDs exhibit a simple device structure and all‐optically controlled operation mode, consuming only 1 nJ of energy for each collision detection. Furthermore, in‐sensor real‐time collision warning is realized by employing a Recurrent Neural Network (RNN) to predict alarm time based on the escape response of the proposed 2D‐CMDs.

Methods and Applications of Space Understanding in Indoor Environment—A Decade Survey

The demand for digitizing manufacturing and controlling processes has been steadily increasing in recent years. Digitization relies on different techniques and equipment, which produces various data types and further influences the process of space understanding and area recognition. This paper provides an updated view of these data structures and high-level categories of techniques and methods leading to indoor environment segmentation and the discovery of its semantic meaning. To achieve this, we followed the Systematic Literature Review (SLR) methodology and covered a wide range of solutions, from floor plan understanding through 3D model reconstruction and scene recognition to indoor navigation. Based on the obtained SLR results, we identified three different taxonomies (the taxonomy of underlying data type, of performed analysis process, and of accomplished task), which constitute different perspectives we can adopt to study the existing works in the field of space understanding. Our investigations clearly show that the progress of works in this field is accelerating, leading to more sophisticated techniques that rely on multidimensional structures and complex representations, while the processing itself has become focused on artificial intelligence-based methods.

The use of pretrained neural networks for solving the problem of reverse searching of X-ray images of prohibited items and substances

The paper considers the application of pretrained neural networks to solve the problem of reverse searching of X-ray images of prohibited items and substances. The purpose of the work is to conduct an analysis and substantiate ways to improve the efficiency of baggage and passenger hand luggage X-ray image recognition systems. An analysis of existing domestic and foreign works in the field of baggage and passenger hand luggage X-ray image recognition is presented. It has been revealed that, despite the achieved results in the development of algorithms for recognizing prohibited items and substances, they do not fully cope with such a complexity factor as the overlay of objects. To solve this problem, the paper proposes to additionally analyze X-ray images with low confidence in object recognition. This stage includes the following steps: image segmentation, extraction of features of segmented image elements; search for similar images in the database; decision-making on the class of segmented image elements. This article discusses the last three steps. Variants of approaches to feature extraction from images are analyzed, particularly those based on the application of convolutional autoencoders and pretrained neural networks. The approach based on the application of pretrained neural networks is chosen. The ResNet-50 architecture neural network, pretrained on the ImageNet collection, is used during the work. In order to apply this model to extract image feature vectors, the last classification layer was preliminarily removed. All the previous layers of the model encode the image into a vector. ResNet-50 generates a 2048-dimensional feature vector of images. The principal component analysis is used to reduce the dimensionality of the image feature vectors. The decision of whether the segmented image element is a prohibited item or substance is considered as a reverse search problem using the k-nearest neighbor algorithm. In this case, the class of the X-ray image element is the class most frequently encountered among the K nearest neighbors. In order to test the proposed approach, a training dataset, including 4,635 images of individual items and substances that may be encountered in baggage and passenger hand luggage, was generated. A comparative analysis of image indexing and image search under different algorithms and feature number is presented. A comparative analysis of the model accuracy is provided. It is concluded that the most acceptable is the “Brute force” algorithm in combination with the principal component analysis.

Pregnancy-induced oxidative stress and inflammation are not associated with impaired maternal neuronal activity or memory function.

Pregnancy is associated with neural and behavioral plasticity, systemic inflammation, and oxidative stress. Yet, the impact of inflammation and oxidative stress on maternal neural and behavioral plasticity during pregnancy are unclear. We hypothesized that healthy pregnancy transiently reduces learning and memory, and these deficits are associated with pregnancy-induced elevations in inflammation and oxidative stress. Cognitive performance was tested using novel object recognition (recollective memory), Morris water maze (spatial memory), and open field (anxiety-like) behavior tasks in female Sprague-Dawley rats of varying reproductive states [non-pregnant (nulliparous), pregnant (near term), and 1-2 months post-pregnancy (primiparous); n = 7-8/group]. Plasma and CA1 proinflammatory cytokines were measured using a MILLIPLEX® magnetic bead assay. Plasma oxidative stress was measured via advanced oxidation protein products (AOPP) assay. CA1 markers of oxidative stress, neuronal activity, and apoptosis were quantified via western blotting. Our results demonstrate CA1 oxidative stress-associated markers were elevated in pregnant compared to nulliparous rats (p ≤ 0.017) but were equivalent levels in pregnant and primiparous rats. In contrast, reproductive state did not impact CA1 inflammatory cytokines, neuronal activity, or apoptosis. Likewise, there was no effect of reproductive state on recollective or spatial memory. Even so, spatial learning was impaired (p ≤ 0.007) while anxiety-like behavior (p ≤ 0.034) was reduced in primiparous rats. Overall, our data suggest maternal hippocampal CA1 is protected from systemic inflammation but vulnerable to peripartum oxidative stress. Peripartum oxidative stress elevations, such as in pregnancy complications, may contribute to peripartum neural and behavioral plasticity.

Neural Technologies for Objects Classification with Mobile Applications

This paper is related to the study of the features of the neural technologies’ application, in particular, ResNet neural networks for the classification of objects in photographs. The work aims to increase the accuracy of recognition and classification of objects in photographs by using various models of the ResNet neural network. The paper analyzes the features of the application of the corresponding models in comparison with other architectures of deep neural networks and evaluates their efficiency and accuracy in the classification of objects in photographs. The process of data formation for training neural networks, their processing and sorting is described. A web application and a mobile application for recognizing and classifying objects in a photo were also developed. A system for classifying objects, in particular airplanes in photographs, was developed using neural network technologies. It gives a recognition and classification accuracy of about 95%. Research results of ResNet models are of great practical importance, as they can improve the classification accuracy of various images. Features of ResNet, such as the use of skip connections or residual connections, make it effective in the relevant tasks. The results of the study will help to implement ResNet in various fields, including medicine, automatic pattern recognition and other areas where the classification of objects in photographs is an important task.

Dexmedetomidine attenuates acute stress-impaired learning and memory in mice by maintaining the homeostasis of intestinal flora

Dexmedetomidine (Dex) has been used in surgery to improve patients' postoperative cognitive function. However, the role of Dex in stress-induced anxiety-like behaviors and cognitive impairment is still unclear. In this study, we tested the role of Dex in anxiety-like behavior and cognitive impairment induced by acute restrictive stress and analyzed the alterations of the intestinal flora to explore the possible mechanism. Behavioral and cognitive tests, including open field test, elevated plus-maze test, novel object recognition test, and Barnes maze test, were performed. Intestinal gut Microbe 16S rRNA sequencing was analyzed. We found that intraperitoneal injection of Dex significantly improved acute restrictive stress-induced anxiety-like behavior, recognition, and memory impairment. After habituation in the environment, mice (male, 8 weeks, 18–23 g) were randomly divided into a control group (control, N = 10), dexmedetomidine group (Dex, N = 10), AS with normal saline group (AS + NS, N = 10) and AS with dexmedetomidine group (AS + Dex, N = 10). By the analysis of intestinal flora, we found that acute stress caused intestinal flora disorder in mice. Dex intervention changed the composition of the intestinal flora of acute stress mice, stabilized the ecology of the intestinal flora, and significantly increased the levels of Blautia (A genus of anaerobic bacteria) and Coprobacillus. These findings suggest that Dex attenuates acute stress-impaired learning and memory in mice by maintaining the homeostasis of intestinal flora.

Maternal high fat diet programs spatial learning and central leptin signaling in mouse offspring in a sex-specific manner

Environmental factors in early life have been demonstrated to increase the risk of neurodevelopmental disorders in offspring, especially the deficiency of the cognitive ability. Leptin has emerged as a key hormone that conveys information on energy stores, but there is growing appreciation that leptin signaling may also play an important role in neurodevelopment. The present study aimed to investigate whether maternal HFD exposure impairs the offspring learning and memory through the programming of central leptin system. We observed that hippocampus-dependent learning and memory were impaired in male but not female offspring from HFD-fed maternal ancestors (C57BL/6 mice), as assessed by novel object recognition and Morris water maze tests. Moreover, the chromatin immunoprecipitation results revealed the maternal HFD consumption led to the increasement in the binding of the histone marker H3K9me3 in male offspring, which mediates gene silencing in the leptin receptor promoter region. Furthermore, there was an increase in the expression of the histone methylase SUV39H1 in male but not female offspring, which regulates H3K9me3. Additionally, it has been observed that IL-6 and IL-1 also could lead to similar alternations when acting on cultured hippocampal neurons in vitro. Taken together, our data suggest that maternal HFD consumption influences male offspring hippocampal cognitive performance in a sex-specific manner, and central leptin signaling may serve as the cross-talk between maternal diet and cognitive impairment in offspring.

Role of the left posterior middle temporal gyrus in shape recognition and its reconstruction during drawing: A study combining transcranial magnetic stimulation and functional near infrared spectroscopy.

There are numerous reports of enhanced or emerged visual arts abilities in patients with semantic impairment. These reports led to the theory that a loss of function on the language side of the brain can result in changes of ability to draw and/or to paint. Further, the left posterior middle temporal gyrus (l-pMTG) has been revealed to contribute to the higher control semantic mechanisms with objects recognition and integration of visual information, within a widely distributed network of the left hemisphere. Nevertheless, the theory has not been fully studied in neural bases. The aim of this study is to examine role of the l-pMTG on shape recognition and its reconstruction within drawing behavior, by using a combining method of the repetitive transcranial magnetic stimulation (rTMS) and functional near-infrared spectroscopy (fNIRS). Eighteen healthy participants received a low frequency inhibitory rTMS to their l-pMTG during the drawing task of the Benton Visual Retention Test (BVRT). There was a significant decrease of the mean accuracy of reproductions in the Complex designs of the BVRT, compared to the Simple and Medium designs. The fNIRS data showed strong negative correlations with the results of the BVRT. Though our hypothesis had a contradiction that rTMS would have inhibited the brain activity in the stimulated site, the results suggest that shape recognition and its reconstruction such as the BVRT require neural activations of the l-TL as well as that of the l-pMTG.

CAST: Cross-Modal Retrieval and Visual Conditioning for image captioning

Image captioning requires not only accurate recognition of objects and corresponding relationships, but also full comprehension of the scene information. However, existing models suffer from partial understanding and object hallucination. In this paper, a Cross-modal retrievAl and viSual condiTioning model (CAST) is proposed to address the above issues for image captioning with three key modules: an image–text retriever, an image & memory comprehender and a dual attention decoder. Aiming at a comprehensive understanding, we propose to exploit cross-modal retrieval to mimic human cognition, i.e., to trigger retrieval of contextual information (called episodic memory) about a specific event. Specifically, the image–text retriever searches the top n relevant sentences which serve as episodic memory for each input image. Then the image & memory comprehender encodes an input image and enriches episodic memory by self-attention and relevance attention respectively, which can encourage CAST to comprehend the scene thoroughly and support decoding more effectively. Finally, such image representation and memory are integrated into our dual attention decoder, which performs visual conditioning by re-weighting image and text features to alleviate object hallucination. Extensive experiments are conducted on MS COCO and Flickr30k datasets, which demonstrate that our CAST achieves state-of-the-art performance. Our model also has a promising performance even in low-resource scenarios (i.e. 0.1%, 0.5% and 1% of MS COCO training set).

Mining local and global spatiotemporal features for tactile object recognition

The tactile object recognition (TOR) is highly important for environmental perception of robots. The previous works usually utilize single scale convolution which cannot simultaneously extract local and global spatiotemporal features of tactile data, which leads to low accuracy in TOR task. To address above problem, this article proposes a local and global residual (LGR-18) network which is mainly consisted of multiple local and global convolution (LGC) blocks. An LGC block contains two pairs of local convolution (LC) and global convolution (GC) modules. The LC module mainly utilizes a temporal shift operation and a 2D convolution layer to extract local spatiotemporal features. The GC module extracts global spatiotemporal features by fusing multiple 1D and 2D convolutions which can expand the receptive field in temporal and spatial dimensions. Consequently, our LGR-18 network can extract local-global spatiotemporal features without using 3D convolutions which usually require a large number of parameters. The effectiveness of LC module, GC module and LGC block is verified by ablation studies. Quantitative comparisons with state-of-the-art methods reveal the excellent capability of our method.